Choreographic inscriptions: A framework for exploring sociomaterial influences on qualities of movement for HCI

With the rise of ubiquitous computing technologies in everyday life, the daily actions of people are becoming ever more choreographed by the interactions available through technology. By combining the notion of inscriptions from actor-network theory and the qualitative descriptors of movement from Laban movement analysis, an analytic framework is proposed for exploring how the interplay of material and social inscriptions gives rise to movement patterns and behaviors, translated into choreographic inscriptions described with Laban effort and shape. It is demonstrated through a case study of an affective gesture mobile device. The framework provides an understanding of (a) how movement qualities are shaped by social and material inscriptions, (b) how the relative strength of inscriptions on movements may change according to different settings and user appropriation over time, and (c) how transforming inscriptions by design across different mediums can generate action spaces with varying degrees of openness

Challenges of developing a digital scribe to reduce clinical documentation burden.

Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout. Advances in artificial intelligence (AI) and machine learning (ML) open the possibility of automating clinical documentation with digital scribes, using speech recognition to eliminate manual documentation by clinicians or medical scribes. However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations. This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts using speech recognition, inducing topic structure from conversation data, extracting medical concepts, generating clinically meaningful summaries of conversations, and obtaining clinical data for AI and ML algorithms

Smartphone-Delivered Ecological Momentary Interventions Based on Ecological Momentary Assessments to Promote Health Behaviors: Systematic Review and Adapted Checklist for Reporting Ecological Momentary Assessment and Intervention Studies.

BACKGROUND: Healthy behaviors are crucial for maintaining a person's health and well-being. The effects of health behavior interventions are mediated by individual and contextual factors that vary over time. Recently emerging smartphone-based ecological momentary interventions (EMIs) can use real-time user reports (ecological momentary assessments [EMAs]) to trigger appropriate support when needed in daily life. OBJECTIVE: This systematic review aims to assess the characteristics of smartphone-delivered EMIs using self-reported EMAs in relation to their effects on health behaviors, user engagement, and user perspectives. METHODS: We searched MEDLINE, Embase, PsycINFO, and CINAHL in June 2019 and updated the search in March 2020. We included experimental studies that incorporated EMIs based on EMAs delivered through smartphone apps to promote health behaviors in any health domain. Studies were independently screened. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed. We performed a narrative synthesis of intervention effects, user perspectives and engagement, and intervention design and characteristics. Quality appraisal was conducted for all included studies. RESULTS: We included 19 papers describing 17 unique studies and comprising 652 participants. Most studies were quasi-experimental (13/17, 76%), had small sample sizes, and great heterogeneity in intervention designs and measurements. EMIs were most popular in the mental health domain (8/17, 47%), followed by substance abuse (3/17, 18%), diet, weight loss, physical activity (4/17, 24%), and smoking (2/17, 12%). Of the 17 studies, the 4 (24%) included randomized controlled trials reported nonstatistically significant effects on health behaviors, and 4 (24%) quasi-experimental studies reported statistically significant pre-post improvements in self-reported primary outcomes, namely depressive (P<.001) and psychotic symptoms (P=.03), drinking frequency (P<.001), and eating patterns (P=.01). EMA was commonly used to capture subjective experiences as well as behaviors, whereas sensors were rarely used. Generally, users perceived EMIs to be helpful. Common suggestions for improvement included enhancing personalization, multimedia and interactive capabilities (eg, voice recording), and lowering the EMA reporting burden. EMI and EMA components were rarely reported and were not described in a standardized manner across studies, hampering progress in this field. A reporting checklist was developed to facilitate the interpretation and comparison of findings and enhance the transparency and replicability of future studies using EMAs and EMIs. CONCLUSIONS: The use of smartphone-delivered EMIs using self-reported EMAs to promote behavior change is an emerging area of research, with few studies evaluating efficacy. Such interventions could present an opportunity to enhance health but need further assessment in larger participant cohorts and well-designed evaluations following reporting checklists. Future research should explore combining self-reported EMAs of subjective experiences with objective data passively collected via sensors to promote personalization while minimizing user burden, as well as explore different EMA data collection methods (eg, chatbots). TRIAL REGISTRATION: PROSPERO CRD42019138739; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=138739

A network model of activities in primary care consultations.

OBJECTIVE:The objective of this study is to characterize the dynamic structure of primary care consultations by identifying typical activities and their inter-relationships to inform the design of automated approaches to clinical documentation using natural language processing and summarization methods. MATERIALS AND METHODS:This is an observational study in Australian general practice involving 31 consultations with 4 primary care physicians. Consultations were audio-recorded, and computer interactions were recorded using screen capture. Physical interactions in consultation rooms were noted by observers. Brief interviews were conducted after consultations. Conversational transcripts were analyzed to identify different activities and their speech content as well as verbal cues signaling activity transitions. An activity transition analysis was then undertaken to generate a network of activities and transitions. RESULTS:Observed activity classes followed those described in well-known primary care consultation models. Activities were often fragmented across consultations, did not flow necessarily in a defined order, and the flow between activities was nonlinear. Modeling activities as a network revealed that discussing a patient's present complaint was the most central activity and was highly connected to medical history taking, physical examination, and assessment, forming a highly interrelated bundle. Family history, allergy, and investigation discussions were less connected suggesting less dependency on other activities. Clear verbal signs were often identifiable at transitions between activities. DISCUSSION:Primary care consultations do not appear to follow a classic linear model of defined information seeking activities; rather, they are fragmented, highly interdependent, and can be reactively triggered. CONCLUSION:The nonlinearity of activities has significant implications for the design of automated information capture. Whereas dictation systems generate literal translation of speech into text, speech-based clinical summary systems will need to link disparate information fragments, merge their content, and abstract coherent information summaries

Design considerations for immersive virtual reality applications for older adults: a scoping review

Immersive virtual reality (iVR) has gained considerable attention recently with increasing affordability and accessibility of the hardware. iVR applications for older adults present tremendous potential for diverse interventions and innovations. The iVR literature, however, provides a limited understanding of guiding design considerations and evaluations pertaining to user experience (UX). To address this gap, we present a state-of-the-art scoping review of literature on iVR applications developed for older adults over 65 years. We performed a search in ACM Digital Library, IEEE Xplore, Scopus, and PubMed (1 January 2010–15 December 2019) and found 36 out of 3874 papers met the inclusion criteria. We identified 10 distinct sets of design considerations that guided target users and physical configuration, hardware use, and software design. Most studies carried episodic UX where only 2 captured anticipated UX and 7 measured longitudinal experiences. We discuss the interplay between our findings and future directions to design effective, safe, and engaging iVR applications for older adults

Conversational Agents for Health and Wellbeing

Conversational agents have increasingly been deployed in healthcare applications. However, significant challenges remain in developing this technology. Recent research in this area has highlighted that: i) patient safety was rarely evaluated; ii) health outcomes were poorly measured, and iii) no standardised evaluation methods were employed. The conversational agents in healthcare are lagging behind the developments in other domains. This one-day workshop aims to create a roadmap for healthcare conversational agents to develop standardised design and evaluation frameworks. This will prioritise health outcomes and patient safety while ensuring a high-quality user experience. In doing so, this workshop will bring together researchers and practitioners from HCI, healthcare and related speech and chatbot domains to collaborate on these key challenges

Conversational agents in healthcare: a systematic review.

Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen's kappa measured inter-coder agreement. Results: The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. Conclusions: The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. Protocol Registration: The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917

Identifying relevant information in medical conversations to summarize a clinician-patient encounter

To inform the development of automated summarization of clinical conversations, this study sought to estimate the proportion of doctor-patient communication in general practice (GP) consultations used for generating a consultation summary. Two researchers with a medical degree read the transcripts of 44 GP consultations and highlighted the phrases to be used for generating a summary of the consultation. For all consultations, less than 20% of all words in the transcripts were needed for inclusion in the summary. On average, 9.1% of all words in the transcripts, 26.6% of all medical terms, and 27.3% of all speaker turns were highlighted. The results indicate that communication content used for generating a consultation summary makes up a small portion of GP consultations, and automated summarization solutions—such as digital scribes—must focus on identifying the 20% relevant information for automatically generating consultation summaries. </jats:p

Envisioning an artificial intelligence documentation assistant for future primary care consultations: A co-design study with general practitioners

Abstract Objective The study sought to understand the potential roles of a future artificial intelligence (AI) documentation assistant in primary care consultations and to identify implications for doctors, patients, healthcare system, and technology design from the perspective of general practitioners. Materials and Methods Co-design workshops with general practitioners were conducted. The workshops focused on (1) understanding the current consultation context and identifying existing problems, (2) ideating future solutions to these problems, and (3) discussing future roles for AI in primary care. The workshop activities included affinity diagramming, brainwriting, and video prototyping methods. The workshops were audio-recorded and transcribed verbatim. Inductive thematic analysis of the transcripts of conversations was performed. Results Two researchers facilitated 3 co-design workshops with 16 general practitioners. Three main themes emerged: professional autonomy, human-AI collaboration, and new models of care. Major implications identified within these themes included (1) concerns with medico-legal aspects arising from constant recording and accessibility of full consultation records, (2) future consultations taking place out of the exam rooms in a distributed system involving empowered patients, (3) human conversation and empathy remaining the core tasks of doctors in any future AI-enabled consultations, and (4) questioning the current focus of AI initiatives on improved efficiency as opposed to patient care. Conclusions AI documentation assistants will likely to be integral to the future primary care consultations. However, these technologies will still need to be supervised by a human until strong evidence for reliable autonomous performance is available. Therefore, different human-AI collaboration models will need to be designed and evaluated to ensure patient safety, quality of care, doctor safety, and doctor autonomy. </jats:sec

A Systematic Review on Healthcare Artificial Intelligent Conversational Agents for Chronic Conditions

This paper reviews different types of conversational agents used in health care for chronic conditions, examining their underlying communication technology, evaluation measures, and AI methods. A systematic search was performed in February 2021 on PubMed Medline, EMBASE, PsycINFO, CINAHL, Web of Science, and ACM Digital Library. Studies were included if they focused on consumers, caregivers, or healthcare professionals in the prevention, treatment, or rehabilitation of chronic diseases, involved conversational agents, and tested the system with human users. The search retrieved 1087 articles. Twenty-six studies met the inclusion criteria. Out of 26 conversational agents (CAs), 16 were chatbots, seven were embodied conversational agents (ECA), one was a conversational agent in a robot, and another was a relational agent. One agent was not specified. Based on this review, the overall acceptance of CAs by users for the self-management of their chronic conditions is promising. Users’ feedback shows helpfulness, satisfaction, and ease of use in more than half of included studies. Although many users in the studies appear to feel more comfortable with CAs, there is still a lack of reliable and comparable evidence to determine the efficacy of AI-enabled CAs for chronic health conditions due to the insufficient reporting of technical implementation details